Difference rewards policy gradients

Abstract Policy gradient methods have become one of the most popular classes algorithms for multi-agent reinforcement learning. A key challenge, however, that is not addressed by many these credit assignment: assessing an agent’s contribution to overall performance, which crucial learning good policies. We propose a novel algorithm called Dr.Reinforce explicitly tackles this combining difference rewards with policy gradients allow decentralized policies when reward function known. By differencing directly, avoids difficulties associated Q -function as done counterfactual (COMA), state-of-the-art method. For applications where unknown, we show effectiveness version learns additional network used estimate rewards.